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Abstract 

A constructive proof is given to the fact that any ergodic Markov chain can be 
, realized as a random walk subject to a synchronizing road coloring. Redundancy 

I (ratio of extra entropy) in such a realization is also studied. 



in 



1 Introduction 



|V] . A random walk in M is a process {Sn)n>o which may be represented as 

Sn = U + U-i + --- + ^i + So, n>l (1.1) 

+^ ■ 

for some sequence (^n)n>i of IID (i.e., independent and identically distributed) random 
variables being independent of 5*0 . Note that equation fll.ip is equivalent to the recursion 
relation 

in ■ Sn=^n + Sn-l, U > 1. (1.2) 

O 

We may introduce a natural analogue of random walk taking values in a finite set V, 
^ ■ say, {1, . . . , m}. Let S denote the set of all mappings of V into itself. A random walk in 
y is a pair of processes {(A„)„>o, (0n)n>i} such that (0n)n>i is a sequence of IID random 
variables taking values in S and being independent of Xq and such that 

K^. X„ = (0„ o o ■ ■ ■ o (/)i)(Ao), n>l. (1.3) 

_c^_' Note that equation (II. 3p is equivalent to the recursion relation 

A„ = MXn-i), n>l. (1.4) 

It is obvious that, for each n > 1, the random variable 0„ is independent of a{Xj,(j)j : 
j < n — 1), since each Xj is measurable with respect to <j{Xq, (pi, ... , (pj). 

It is now natural to extend the index set to Z, the set of all integers, as follows. 
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Definition 1.1. A random walk in V parametrized by Z is a pair of processes 
which satisfies the following conditions: 

(i) (0n)nez is a sequence of IID random variables taking values in S; 

(ii) for each n G Z, the random variable (pn is independent of cr(Xj, (pj : j < n — 1); 

(iii) Xn = (f>n{,Xn~i) holds almost surely for all n E Z. 

If 0„'s have common law /i on S, such a random walk is called a ^-random walk. 

Our /i-random walk may also be called a random walk in a finite directed graph subject 
to a road coloring. The reason will be explained in Section O Each element of V will be 
called a site. 

If a yU-random walk (0„)nGz} is given, then the process is a Markov 

chain in V whose one-step transition probability is given by 

P(Xi = y\XQ = x) = ^i{a eH: a{x) = y) , x,yeV. (1.5) 

Conversely, if a Markov chain {Yn)n& is given, we call /i a mapping law for the Markov 
chain if the following identity holds: 

P(Fi = y\Yo = x)=fi{creE: a{x) = y) , x,yeV. (1.6) 

In this case, for any /x-random walk {(X„)„ga, (0n)nGz}, we can easily show that the 
Markov chain {Xn)n& is identical in law to {Yn)nez- 

Proposition 1.2. For any Markov chain {Yn)n& in V , there exists a mapping law /i. 

A proof by means of rational approximation can be found in [9]. We shall give its 
constructive proof in the next section. 

Note that, for a given Markov chain, there may exist several mapping laws. We may 
expect that we can take a nice mapping law in the following sense. 

Definition 1.3. Let /i be a probability law on S and denote by Supp(/i) the support of /x. 
We say that /i is synchronizing (or simply sync) if there exists a finite sequence ai, . . . , cr^ 
of elements of Supp(/i) such that ap o ap^i o ■ • ■ o ai maps V into a signleton. 

Note that a /^-random walk associated with a sync mapping law is utilized in Propp- 
Wilson's sampling method of stationary law, which is called coupling from the past] we 
shall mention it briefiy in Section HJ 

Suppose that is sync and let {{Xn)n&, {(pn)n&z} be a /x-random walk. We may 
assume without loss of generality that for any x E V there exists a G Supp(/i) such that 
X G <y[V); in fact, the Markov chain {Xn)n& never visits such sites x E V that x ^ <y{V) 
for any a G Supp(/i). Then we see that the Markov chain {Xn)nez is ergodic, i.e., the 
following two conditions hold (see, e.g., |5]): 
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(i) the Markov chain is irreducible, i.e., P{Xq = x) > for all x & V and for any 
x,y & V there exists n > 1 such that = y\XQ = x) > 0; 

(ii) the Markov chain is aperiodic, i.e., for any x ^ V, the greatest common divisor of 
{n>l: P{Xn = x\Xo = x) > 0} is one. 

The condition (i) is obvious. The condition (ii) may be verified as follows. Let x & V. Take 
a E V such that fXp o ■ ■ ■ o aiiV) = {a} and take g > 1 such that P{Xg = x\Xq = a) > 0. 
Then the set {n > 1 : P{Xn = x\Xo = x) > 0} contains all integers greater than p + q, 
and hence its greatest common divisor is one. 

The following theorem asserts that the converse is also true. 

Theorem 1.4 (^). Suppose that {Yn)n<^'L is an ergodic Markov chain. Then there exists 
a sync mapping law. 

To prove Theorem 11.41 the authors in [H] utilized a profound graph-theoretic theorem, 
which was recently obtained by Trahtman [7], the complete solution to the road color- 
ing problem; we shall explain it briefly in Section [51 In this chapter, we would like to 
give an elementary, self-contained and constructive proof of Theorem 11.41 without using 
Trahtman's theorem. 

The remainder of this chapter is as follows. In Section [21 we give constructive proofs 
to Proposition 11.21 and Theorem II. 4[ In Section [3], we study redundancy in random walk 
realization of a Markov chain. In Section [H we mention the couplinng from the past. In 
Section \5[ we explain how our random walk is related to road coloring. In Section ^ we 
provide a summary and conclusion. 

2 A constructive proof of existence of sync mapping 
law 

A matrix Q = {qx,y)x,y£V with non-negative entries is called a transition matrix if 



We give a constructive proof of Proposition 11.21 for later use. 

A constructive proof of Proposition It suffices to show that, for any transition matrix 
Q, there exists a mapping law /i for Q, i.e.. 




for all X G V. 



(2.7) 



y&V 



fi{a eT.: a{x) = y) 



x,y G V. 



(2.8) 



We define 



E{Q) = {ix,y)eVxV:qx,y>0}. 



(2.9) 
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Let us prove the result by induction of ]^E{Q), where jjA stands for the number of elements 
of A. It is obvious by fl2.7l) that ]^{y & V : q^y > 0} > 1 for all x & V, and hence that 

MQ) > ^v. 

Suppose that tl^(Q) = \\V. Then, by ([22D, it holds that \\{y e V : q^^y > 0} = 1 for all 
X & V. This shows that there exists a G S such that 

{1 if w = (t(x), 
^ ^ ^' (2.10) 
otherwise. 

Thus the Dirac mass at a is as desired. 

Let k > and suppose that all transition matrix Q such that ]^E{Q) < k admits a 
mapping law. Let Q be a transition matrix such that ]^E{Q) = k. Write 

e = mm{q^^y : (x, y) G E{Q)}. (2.11) 

Since tl-£'(Q) > tl^; we see that < £ < 1. Take {x,y) G E{Q) such that e = q^^y, and 
take a G S such that (x, o"(a;)) G £'((5) for all x E V. Define Q = {qx,y)x,y&v by 

qx,y = - ^^l{<7(x)=y}) • (2.12) 

Then we see that Q is a transition matrix and that ^E{Q) < k. Now, by the assumption 
of the induction, we see that Q admits a mapping law Jl. Therefore we conclude that 
(1 — e)Jl + eSa is a mapping law for Q. The proof is now complete. □ 

Utilizing Proposition II. 2[ we give a constructive proof of Theorem 11.41 



A constructive proof of Theorem \1.4\ Let Q be the transition matrix of an ergodic Markov 
chain. Then we see that there exists r > 1 such that the r-th product Q^' has positive 
entries. 

Take xq E V arbitrarily and set Vr = {xo}. If Vk is defined for k = r,r — 1, ... ,1, define 
Vk-i recursively by 

Vk-i ={xeV -.{x^y) e E{Q) for some y e Vk} , (2.13) 
where E{Q) has been defined in (12.91) . Note that 

UK < tlK-i < ■ • • < UK). (2.14) 

Since has positive entries, we see that Vq = V. 

For k = r,r — 1, . . . ,1, we pick ct/j G S so that <7k{x) G if x G \4_i and (x, <Jk{x)) G 
E{Q) if X ^ Vk-i. We then have 

CTr O (Tr-l O ■ ■ ■ O ai{V) = {Xq} . (2-15) 
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Let n^^^ denote the uniform law on {(Xi, . . . , cxr}. Then we see that jj^^^ is a sync mapping 
law for a transition matrix Q'^'^'^ = {qx^l)x,y£V where 

1 

q^l = f^^'K^ e S : y = a{x)) = -J2 l{.=<x.(x)}- (2.16) 

A;=l 

Write 

e = min{g,,^ : (x, y) G E{Q)} > (2.17) 
and define Q"^^) = {q^xl)x,yev by 

= Y^(^-.^-^^S)- (2-18) 

Then Q^"^^ is a transition matrix, so that we may obtain a mapping law /i^^-* for Q*-^-* in 
the constructive way of the proof of Proposition 11.21 given above. 

Now we define 

= + (1 - £)/i(2\ (2.19) 

which we have proved that is a sync mapping law for Q. The proof is therefore complete. 

□ 



3 Redundancy in random walk realization 

The uncertainty associated with information source may be measured by entropy (see, 
e.g., [2]). A Markov chain Y = (l^)nez with transition matrix Q = {qx^y)x^y^y and with 
stationary law A has its entropy given by 

h(y) = - ^ Hx)q^^y\ogq^^y, (3.20) 

where we adopt the binary logarithm log = logj for simplicity, and follow the usual 
convention: OlogO = 0. For a probability law /i on S, an IID sequence = {4>n)n&z with 
common law fi has its entropy given by 

h{<P) = h{fi) = -^/i(a)log/i(^)- (3.21) 

A yU-random walk (X, 0) = {{Xn)n£Z, {<Pn)n&} with stationary law A is a Markov chain 
whose transition matrix Q = {q(^x,y),{y,a)) (x,u),{y,a-)ev xs and stationary law A given by 

Q(x,uUy,a) = K(^)My=a{x)}, K{x, u)) = fl{iy)X{w E V : X = u{w)). (3.22) 
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Now its entropy h{X, 0) is computed as 

h{X,(f))=- X{x,^))q^x,u),(y,^)'^^SqixM)M (3-23) 

(x,!/),(j/,o-)gVxS 



X = u{w)) \ l{y=^(^)}/i(a) log/i((T) (3.24) 

(3.25) 



J2 |$^M^)A(«;Gl^: 

x,yeV, o-gS I vGS 

^ A(x)<^ ^1{2,=<,(^)} I ^((T)log/i(a 

5^A(x)li5^^(a)log/i(^)[ (3.26) 

= -5^M^)log/x(a). (3.27) 

Thus we obtain h{X, (p) = h{(f)) = h{fi). 

If the Markov chain Y is identical in law to X for some /x-random walk (X, 0), we have 

hifi) > h{Y). (3.28) 

In fact, by (12. 8p . we have /i((T) < q^^y if y = cr(x), and hence we see that 

^(/^) = - 5Z A(a;) 5^ /"(^) log /"(^) (3-29) 

x€V o-GE 

= -5^A(x) ^ M^)log/i(^) (3.30) 

>-^A(x) fi{(r)\ogq^^y (3.31) 

= - Y Hx)qx,y^ogq^^y = h{Y). (3.32) 

The inequality fl3.28p shows that any /x-random walk realization of Y requires some extra 
entropy, the extent of which may be measured by 

This quantity r(/i; Y) is called the (relative) redundancy in the /i-random walk realization 
of the Markov chain Y. We denote the totality of all possible redundancies by 

p{Y) = {r(/i; Y) : /x is a mapping law for Y}. (3.34) 

Theorem 3.1. For a Markov chain Y , the following assertions hold: 

(i) the set p{Y) has finite minimum r{Y) > and maximum R{Y) < 1; 



6 



(a) for any r{Y) < r < R{Y), there exists a mapping law /i for Y such that r{^; Y) 



r. 



Moreover, if Y is ergodic, then the following assertion also holds: 

(Hi) for any r{Y) < r < R{Y), there exists a sync mapping law fi for Y such that 
r{fi] Y) = r. 



Proof. Let us remark on several basic facts about the entropy. Since S is a finite set, the 
totahty of probabihty measures on S, which is denoted by 'P(S), is equipped with the 
total variation topology. It is well-known that 'P(S) is compact and that /i„ — )■ /i if and 
only if — )■ /i((j) for all cr G S. By definition fl3.2ip . the function 'P(S) 9 /i H- h{jj) 
is continuous. 

(i) Let V{Y) denote the set of all mapping laws for Y . It is obvious that V{Y) is 
a compact convex subset of 'P(S). Since h{fi) > h{Y) > for all G 'P(^), and since 
t (t — h(Y))/t is continuous in t > h(Y), we see that V{Y) 3 ji y-^ r{jj,;Y) is continuous. 
Hence we see that the set piY) has finite minimum r{Y) and maximum R{Y). 

(ii) Take g V{Y) such that r{Y) = r(/x(i);r) and R{Y) = r{fi^^^;Y). Let 
< p < I. Then /ip := pfx^^^ + {l—p)n^'^^ also belongs to V(Y). Since [0, 1] 3 p ^-^ ^{l^p] 

is continuous, we see that p{Y) contains all r such that r{Y) < r < R{Y). Thus we obtain 
(n). 



(iii) Suppose that Y is ergodic. Theorem 11.41 implies that there exists a sync mapping 
law for Y. Let r(Y) < r < R{Y) and take r^^\ r*^^^ such that r(Y) < r^^'^ < r 
and r < r^^^ < RiY). By (ii), we may take mapping laws yu'-^^ and /i*^^-* for Y such that 
r{n^^^; Y) = r^^^ and r{ij,^'^^; Y) = r^^\ Now we may take e > small enough such that 



r((l - + r) < r < r((l - e)fi^^^ + e/i^"); Y). 

Hence we may take < p < 1 such that the mapping law p defined by 

/i = (1 - + (1 - + 



(3.35) 



(3.36) 



satisfies r(/i; Y) = r. This shows that is a sync mapping law for Y. Therefore the proof 
is complete. □ 



Example 3.2. Let V = {1,2,3} and let 

Q- 



9i,2 qi,3 

C[2,l 12,2 12,3 
Q3,l 13,2 C[3,3 



■ 1 " 

1/2 1/2 
1/2 1/2 



(3.37) 



The Markov chain Y with transition matrix Q has a unique stationary law 

A= [A(1),A(2),A(3)] =i [3,2,4]. 



(3.38) 
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A simple computation leads to h{Y) = 2/3. For a mapping law /i for Y, elements which 
may possibly be contained in Supp(/i) are the following four: 





"0 





r 




'0 





1" 




'0 





1" 




"0 





1" 




1 
















1 










1 




1 










1 













1 







1 













1 






(3.39) 

Set p = jj,{a^^^). A simple computation leads to 

= p, ^(^(3)) = = 1/2 - p. (3.40) 

Thus we obtain 

/i(^) = 2/(p) + 2/(1/2 -p), (3.41) 

where f{t) = —tlogt. Since the variable p may vary in [0, 1/2], we see that h{fj,) ranges 
[1,2], where the minimum /i(/i) = 1 is attained at p = and 1/2 and the maximum 
/i(/x) = 2 at p = 1/4. Hence we obtain 

r{Y) = 1/3, R{Y) = 2/3. (3.42) 

In this case, for all p G [0, 1/2], the mapping law /x is sync; in fact, a^^^ o a^'^\V) = {1} 
and ct(3) o o o a(3)(V) = {3}. 

Before closing this section, we mention the following theorem, which provides a neces- 
sary and sufficient condition for zero minimum redundancy. 

Theorem 3.3 ([9]). Suppose that Y is ergodic. Then r{Y) = if and only if Y is 
p-uniform, i.e., there exists a probability law u on V and a family {r^ : x G V} of 
permutations of V such that 

qx,y = i^ir^iy)), x,yeV. (3.43) 
For the proof of Theorem 13. 3[ see [9]. 

4 Coupling from the past 

In some practical problems, we sometimes need to simulate the stationary law of an ergodic 
Markov chain. As a powerful method for the simulation, Propp-Wilson's coupling from 
the past is widely known; see [6] and also [3] and [1]. The fundamental idea is to utilize 
a random walk realization associated with a sync mapping law. Let us explain it briefly. 

Let an ergodic Markov chain be given and suppose that we find a sync mapping law 
fi for the Markov chain. Then a /i-random walk {(^n)neZ) (0n)nGz} is a realization of 
the Markov chain. Let (ai, . . . , cXp) be a finite sequence of elements of Supp(/i) such that 
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CTp o ■ ■ ■ o (Ti{V) is a singleton. The latest time when the exact sequence (cjp, o"p_i, . . . , ai) 
can be found in (0o, • • •) will be denoted by 

T = sup{A; eZ: 0>k + p-l, (pk+p-i = (^p, ■ ■ ■ , 4>k = en}- (4.44) 

Here we understand that sup0 = — oo. Note that T is finite almost surely. This random 
time T plays a role of stopping time in the sense that 

{T = k} ea{(j)o,^-i,...,^k) forO>fc + p-l. (4.45) 

Since dp o ■ ■ ■ o ai (V) is a singleton, we see that 0o ° 0-i o • ■ ■ o 0^. maps into a singleton. 
Thus it holds that 

Xq = (f)QO (f)_i o ■ ■ ■ o 0r(x) a.s. (4.46) 

for all X & V. This shows the foUoiwng: We pick a sequence /o, . . . from the law 
fi up to the latest time T when {fx+p-i, ■ ■ ■ , fx) = (o"p, • • • , o"i)- Then the resulting site 
/o o o ■ ■ ■ o fxix), which does not depend on the choice of x G V^, is a sample point 
from the stationary law, which is as desired. 

This method can be applied to simulate a Gibbs distribution. In this case, a sync 
mapping law can be constructed with the help of monotonicity structure of the state 
space V. 

Remark 4.1. The identity fl4.46p implies that, for each n E 1^, the random variable Xn 
is measurable with respect to a{(f)j : j < n). One can ask what happens when fj, is not 
sync. The following theorem answers this question. 

Theorem 4.2 (Yano [8j). Suppose that the Markov chain {Xn)n&i is ergodic. and that fi 
is not sync. Then, for each n G Z, the random variable Xn is not measurable with respect 
to a{(j)j : j < n). 

For the proof of Theorem 14. 2[ see [H| • 



5 Random walk and road coloring 

Let us explain how our /i-random walk is related to road coloring. 

First, let us introduce some notations in graph theory. A finite directed graph is the 
pair (y. A) of finite sets V and A associated with mappings i : A ^ V and t : A ^ V . 
Each element of V will be called a site (or a node) and each element a of A will be called 
a (oneway) road (or an arrow) which runs from i{a) to t{a). For a G A, the site i{a) (resp. 
t(a)) will be called the initial (resp. terminal) site of a. For x G V^, the number of roads 
running from x, namely, 

0{x) = i{ae A:i{a) =x} , (5.47) 
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will be called the outdegree at the site x. If 0{x) does not depend on x eV, the directed 
graph (V, A) is called of constant outdegree. A path from x^Vtoy^Visa word 
w = (ai, . . . , a^i) of roads such that ai runs from x to 2(02), a2 to 2(03), . . . , to i{an), 
and to y. The number = is called the length of the path w = (ai, . . . , a„). The 
directed graph (V, A) is called strongly connected if, for any a;, y G l^, there exists a path 
from a; to y. The directed graph (V, A) is called aperiodic if, for any x G the greatest 
common divisor of the set of L{w) > 1 among all paths w from x to itself is one. 

Second, we introduce some notations in road coloring. Suppose that (V, A) is of con- 
stant outdegree and denote the common outdegree by d. A road coloring of (V, A) is a 
partition of A into d disjoint subsets C = {c^^\ . . . , c^'^^} such that, for each x G V, each 
color c'-'^^ contains one and only one road whose initial site is x. For a finite sequence 
s = (ci, . . . , Cp) of elements of C, a path w = (ai, . . . , a^) is said to be along s if E Ck 
for all A; = 0, 1, ... ,p. The following notion originates Adler, Goodwyn and Weiss [1]. 

Definition 5.1. A road coloring C of (V, A) is called sync if there exists a finite sequence 
ci, . . . , Cp of elements of C such that all paths along (ci, . . . , Cp) have common terminal 
site. 

Let us give an example. 

Example 5.2. Let V = {1, 2, 3} and A = {a(^''=) : x G V, k = 1,2} and define the initial 
and terminal sites of each road as follows: 



a 














i{a) 


1 


2 


3 


1 


2 


3 


t{a) 


3 


3 


1 


3 


1 


2 



Take the road coloring C = {c^^\ c'-^^} defined by 

c(i) = : X e V}, c(2) = {a("'2) -.xeV}. (5.49) 

Now it is obvious that the road coloring C is sync; in fact, all paths along {c^^\ c^'^\ c^'^\ c^^-*) 
have common terminal site 3. 

Third, we recall the road coloring problem. If a directed graph (V, A) of constant outde- 
gree admits a sync road coloring, then it is necessarily strongly connected and aperiodic. 
The converse was posed as a conjecture by Adler, Goodwyn and Weiss [T], which had 
been called the road coloring problem until it was completely solved by Trahtman [7j. 

Theorem 5.3 (Trahtman [7]). A directed graph which is of constant outdegree, strongly 
connected, and aperiodic, does admit a sync road coloring. 

Fourth, let us explain how to understand our /i-random walk by means of road coloring. 
Let /i be a probability law on E. Since E is a finite set, the support of fi may be written 
as {(y^^\ . . . , cr*-'^^}. We define the set A of roads as the totality of a*-^''^^ for x G V and 
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k = 1, . . . ,d where a*^^''^^ runs from x to a^''\x). Thus the law /i induces naturally the 
road coloring C = {c^^\ . . . , c^'^^} such that 

= {a("'^) -.xeV}. (5.50) 

It is now obvious that the probability law fi is sync in the sense of Definition 11.31 if and 
only if the road coloring C is sync in the sense of Definition 15. 1[ 

For a /i-random walk (X, 0), the process X moves from site to site in the directed graph 
(y, A) via the equation X„ = 0„ (X„_i), being driven by the colors of roads indicated by 
which are randomly chosen from the road coloring C induced by /i. Thus we may call 
{X, 0) a /i-random walk in the directed graph (V, A) subject to the road coloring C. 

Let y be a Markov chain and suppose that Y is realized as X of a /x-random walk 
(X, 0) in the directed graph (X, 0) subject to the road coloring C induced by /i. Then, to 
each edge {x, y) G E{Y), there corresponds at least one road a which runs from x to y. For 
example, consider Example 13.21 with p = 0. In this case, we have Supp(/i) = {a^^\ a^^^, 
and 

E{Y) = {(1, 3), (2, 1), (2, 3), (3, 1), (3, 2)}. (5.51) 

The set A of roads induced by /i is given as A = {a*^^'^-* : x E V,k = 1,2}, where the 
initial and terminal sites of each road are given as f l5.48p . Then we find that the road 
coloring induced by /i is nothing else but C = {c^^\c^'^^ given as fl5.49p in Example 15. 2[ 
where we note that a^^^ and a^'^^ correspond to c'-^^ and c*-^\ respectively. Note that there 
exist two roads a*^^'^-* and a'-^'^-' which run from 1 to 3, which are colored differently from 
each other. See Figure 1 below for the illustration. 




Figure 1. 



6 Conclusion 

We have introduced a random walk in a finite set as a stochastic evolutionary process 
driven by an IID sequence of mappings. It can be understood as a random walk in a 
finite directed graph moving according to random road colors. Any ergodic Markov chain 
is proved to be realized, in a constructive way, by a random walk associated with a sync 
mapping law. The redundancy in random walk realization with a sync mapping law can 
be as close as desired to the minimum redundancy. 
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